Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: improved support and testing for nested codeblocks #257

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ErikBjare
Copy link
Owner

@ErikBjare ErikBjare commented Nov 15, 2024

Fixing the issue I wrote about in #251 (comment)


Important

Improves nested code block handling in _extract_codeblocks() and adds tests for nested and multiple markers in tests/test_codeblock.py.

  • Behavior:
    • Improved handling of nested code blocks in _extract_codeblocks() in gptme/codeblock.py.
    • Supports multiple code block markers on the same line.
  • Testing:
    • Added test_extract_codeblocks_nested_oneline, test_extract_codeblocks_complete_nested, and test_extract_codeblocks_multiple_nested in tests/test_codeblock.py to test nested code block extraction.
    • Ensures correct extraction of nested blocks and multiple markers on the same line.

This description was created by Ellipsis for 648322b. It will automatically update as commits are pushed.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Reviewed everything up to 648322b in 21 seconds

More details
  • Looked at 133 lines of code in 2 files
  • Skipped 0 files when reviewing.
  • Skipped posting 0 drafted comments based on config settings.

Workflow ID: wflow_5HcG2RIHKXsOmCGd


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@@ -66,22 +66,41 @@ def _extract_codeblocks(markdown: str) -> Generator[Codeblock, None, None]:
# TODO: fix to actually be correct
start_idx = sum(len(line) + 1 for line in lines[:idx])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The calculation of start_idx is incorrect. It should account for the position of the opening backticks, not just the start of the line. Consider adjusting the calculation to accurately reflect the start of the code block.

Suggested change
start_idx = sum(len(line) + 1 for line in lines[:idx])
start_idx = sum(len(line) + 1 for line in lines[:idx]) + line.find('```')

@codecov-commenter
Copy link

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
84 2 82 0
View the top 2 failed tests by shortest run time
tests.test_reduce::test_reduce_log
Stack Traces | 0.662s run time
@pytest.mark.slow
    def test_reduce_log():
        msgs = [
            Message("system", content="system prompt"),
            Message("user", content=" ".join(fn.name for fn in [readme, cli, htmlindex])),
            long_msg,
        ]
        len_pre = len_tokens(msgs)
        print(f"{len_pre=}")
    
        limit = 1000
        reduced = list(reduce_log(msgs, limit=limit))
        len_post = len_tokens(reduced)
        print(f"{len_post=}")
        print(f"{reduced[-1].content=}")
    
>       assert len_pre > len_post
E       assert 4733 > 4733

tests/test_reduce.py:50: AssertionError
tests.test_reduce::test_truncate_msg
Stack Traces | 1.29s run time
def test_truncate_msg():
        len_pre = len_tokens(long_msg)
        truncated = truncate_msg(long_msg)
>       assert truncated is not None
E       assert None is not None

tests/test_reduce.py:26: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
Got feedback? Let us know on Github

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants